Lens flare is the light scattered in lens systems through internal reflections in the lens. Flare may be caused by a very bright light source (e.g., the Sun or a spotlight) that is either in the image or shining at the lens from a particular angle. The appearance of flare in natural images is determined by the shape of the camera aperture and the position of the bright light source, and it is often expected to have an almost regular shape, e.g., rings, circles, straight lines across the image, etc., although, in certain circumstances, a flare may be spread across a portion of an image (such as haze, which may affect large image regions with slowly varying strengths).
Taking high quality photographs in low ambient light conditions, or photographing dynamic scenes (e.g., sport scenes) may be challenging, e.g., due to camera motion and/or the motion of objects within a scene during image capture. One way to reduce motion blur without amplifying an image's noise is to capture and fuse multiple short-exposed images of the scene. Such operations are often called “Still Image Stabilization” (SIS). While shortening image exposure times can reduce motion blur artifacts, it may do so at the expense of a noisier and/or darker image.
One common approach to image fusion consists of: (1) selecting a “reference image” from a set of captured images; (2) globally registering each of one or more “non-reference” images with respect to the reference image; and (3) synthesizing an output image by fusing the non-reference images to the reference image. In this way, the output image represents the scene as it was at the time the reference image was captured, while the one or more non-reference images improve or otherwise alter the reference image. As an example, image fusion may be used to reduce the noise in the reference image, e.g., by averaging/merging multiple observations of each reference pixel across all images, or may be used to enhance details found in the reference image.
One common approach to synthesizing an output image by fusing all registered non-reference images to the reference image is to directly average corresponding pixels from across the images. Direct averaging may reduce the noise in the stationary areas of the image, but it may also introduce ghosting artifacts. Ghosting artifacts can occur when some of the pixels in the reference image are occluded in some of the non-reference images, e.g., due to moving objects in the scene. When there is motion between the captured images, significant ghosting artifacts may be present in the final output when the images are directly averaged.
In addition to differences between the reference and non-reference images caused by motion, there may also be differences caused by foreign light that is present in one or more of the images, but not present in the other images. One common source of foreign light in an image is so-called lens flare, referred to above.
The advent of small, mobile, multipurpose devices such as smartphones and tablet devices has resulted in a need for high-resolution, small form factor cameras, capable of generating high levels of image quality, for integration in the mobile devices. Increasingly, as users rely on these multifunction devices as their primary cameras for day-to-day use, users demand features, e.g., zoom photography, which they have become accustomed to using in dedicated-purpose camera bodies. The zoom function may be useful for capturing the details of a scene or alternatively capturing the context in which those details exist. The ability to change focal length to achieve zoom effects is sufficiently compelling to users of dedicated purpose cameras that it compels them to carry bags with an array of removable lenses, each of which may weigh more and takes up more space than many common examples of a multifunction device, such as a mobile phone.
Providing the zoom feature in a camera unit of a multifunction device has traditionally required moving mechanical parts that increase complexity and cost of the device. Such moving parts may also reduce reliability of the device and take up valuable space inside the device, which puts the desire for zoom functions in direct conflict with the desire for smaller camera units that take up less space in the multifunction device.
Thus, in some imaging devices, e.g., those described herein, there may be two (or more) optical sensors/camera units, e.g., which are configured to capture images of a scene, but which may have different specifications or operating parameters, such as a focal length, zoom, field of view, exposure settings, etc. As mentioned above, fused images may be created by blending together images or portions of images of a plurality of images captured by an imaging device having multiple optical sensors/camera units. Foreign light effects, e.g., in the form of lens flare patterns found in images captured by the various camera units of a multi-camera unit image capture device may be different in each of the captured images, due to, e.g., difference in lens design, sensor, position, and/or the angle between the optical axes of the multiple cameras and the bright light source(s) that caused the lens flare to occur in one (or more) of the captured images. Consequently, a set of images captured for image fusion purpose may be affected by flare in different ways depending on the mismatch of the flare patterns in each image. Fusing such images together may result in artifacts like unnatural jaggy flare borders, loss of image detail, hazy patches, unnatural looking flare borders, non-photorealistic flare composition, and/or skewed image coloration.